Digital video consumes large amounts of storage and transmission capacity. A typical raw digital video sequence includes 15 or 30 frames per second. Each frame can include tens or hundreds of thousands of pixels (also called pels). Each pixel represents a tiny element of the picture. In raw form, a computer commonly represents a pixel with 24 bits. Thus, the number of bits per second, or bitrate, of a typically raw digital video sequence can be 5 million bits/second or more.
Most computers and computer networks lack the resources to process raw digital video. For this reason, engineers use compression (also called coding or encoding) to reduce the bitrate of digital video. Compression can be lossless, in which quality of the video does not suffer but decreases in bitrate are limited by the complexity of the video. Or, compression can be lossy, in which quality of the video suffers but decreases in bitrate are more dramatic in subsequent lossless compression. Decompression reverses compression.
In general, video compression techniques include intraframe compression and interframe compression. Intraframe compression techniques compress individual frames, typically called I-frames, key frames, or reference frames. Interframe compression techniques compress frames with reference to preceding and/or following frames, and are called typically called predicted frames, P-frames, or B-frames.
Many intraframe and interframe compression techniques are block-based. A video frame is split into blocks for encoding. For example, an I-frame is split into 8×8 blocks and the blocks are compressed. Or, a P-frame is split into 16×16 macroblocks (e.g., with 4 8×8 luminance blocks and 2 8×8 chrominance blocks) and the macroblocks are compressed. Different implementations can use different block configurations.
Standard video encoders experience a dramatic degradation in performance when the target rate falls below a certain threshold. For block-based video compression and decompression, quantization and other lossy processing stages introduce distortion that commonly shows up as blocking artifacts—perceptible discontinuities between blocks. At low bitrates, high frequency information for the blocks of I-frames may be heavily distorted or completely lost. Similarly, high frequency information for the residuals of blocks of P-frames (the parts of the P-frames not predicted by motion estimation or other prediction) may be heavily distorted or completely lost. As a result, significant blocking artifacts can arise in “low-pass” regions, and cause a substantial drop in the quality of the reconstructed video.
Some previous encoders attempt to reduce the perceptibility of blocking artifacts by processing reconstructed frames with a deblocking filter. The deblocking filter smoothes the boundaries between blocks. While the deblocking filter can improve perceived video quality, it has several disadvantages. For example, the smoothing occurs only on reconstructed output in the decoder. Therefore, the effect of deblocking cannot be factored into the process of motion estimation, motion compensation or transform coding for a current frame, even when in-loop deblocking is being used. On the other hand, the smoothing of the current frame by the post-processing filter (i.e., out-of-loop deblocking) can be too extreme, and the smoothing process introduces unnecessary computational complexity.
Given the critical importance of video compression and decompression to digital video, it is not surprising that video compression and decompression are richly developed fields. Whatever the benefits of previous video compression and decompression techniques, however, they do not have the advantages of the following techniques and tools.